The computer industry is moving toward fast, packetized, serial input/output (I/O) bus architectures, in which computing hosts and peripherals are linked by a switching network, commonly referred to as a switching fabric. A number of architectures of this type have been proposed, culminating in the “InfiniBand™” (IB) architecture, which has been advanced by a consortium led by a group of industry leaders (including Intel, Sun Microsystems, Hewlett Packard, IBM, Compaq, Dell and Microsoft). The IB architecture is described in detail in the InfiniBand Architecture Specification, Release 1.0 (October, 2000), which is incorporated herein by reference. This document is available from the InfiniBand Trade Association at www.infinibandta.org.
A host processors (or host) connects to the IB network via a network adapter, which is referred to in IB parlance as a host channel adapter (HCA). When an IB “consumer,” such as an application process on the host, needs to open communications with some other entity via the IB network, it asks the HCA to provide the necessary transport service resources by allocating a queue pair (QP) for its use. Each QP is configured with a context that includes information such as the destination address (referred to as the local identifier, or LID), service type, and negotiated operating limits. Communication over the network takes place between a source QP and a destination QP, so that the QP serves as a sort of virtual communication port for the consumer. To send and receive communications over the network, the consumer initiates work requests (WRs), which causes work items, called work queue elements (WQEs), to be placed onto the appropriate queues. The channel adapter then executes the work items, so as to communicate with the corresponding QP of the channel adapter at the other end of the link. The IB specification permits the HCA to allocate as many as 16 million (224) QPs, each with a distinct queue pair number (QPN). A given consumer may open and use multiple QPs simultaneously.
Typically, a range of addresses in the memory space of the host is assigned to the HCA for use as “doorbells” by host processes in accessing their allocated QPs. Each QP is associated with a different doorbell, which is mapped into a separate page (typically a 4 KB segment) of the memory space. When a user process wishes to submit a WR to one of its QPs, it writes to the doorbell, which prompts the HCA to service the read or write request. The operating system (OS) on the host processor is responsible for ensuring that each process writes only to the memory pages that are assigned to it. This type of memory protection is standard in all operating systems known in the art. Together with the doorbell assignment, it ensures that each process can access only its own assigned QPs. Since each doorbell consumes a page of memory, however, an enormous virtual memory space is required in order to support the full complement of 16 million QPs provided by the IB standard. It is known in the art to allocate doorbells smaller than a full page, and thus to reduce the memory range required for this purpose, but this mechanism offers at best a partial solution to the problem of supporting a large number of QPs.